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The purpose of this study was to measure the effect of the Baylor College of Medicine Summer Science Institute 
(BCMSSI) on teacher science content knowledge and students’ science performance. The analysis of the effect of 
participation showed an increase of 253 scale score points on the 2015 STAAR science performance of eighth- 
grade students whose teachers completed all 60 professional development (PD) hours. A higher proportion of 5" 
and 8"-grade students whose teachers completed the full PD hours met Satisfactory at the Level IT phase-in 1 
standard and Advanced performance standard on the 2015 STAAR science test compared to their peers whose 
teachers did not complete the full PD hours. The results also showed statistically significant moderate to large 
gains in teacher content knowledge and self-efficacy at all grade levels. Teachers at all grade levels were also, 
overwhelmingly, positive about the science institute and agreed that the resources and materials that they received 
were useful. Increasing the number of teachers who complete the BCMSSI program should be considered. 
Continued instructional focus should be placed on improving the science performance of at-risk and economically- 
disadvantaged students. 


workshop sessions throughout the academic year. The 
program culminated with a science festival in which 
participants displayed skills and content learned. 

Baylor College of Medicine was the contracted 
service provider. The contract was financed under the 
Teacher Incentive Fund (TIF) grant in the amount of 
$285,000. In addition, the cost of the follow-up 
workshops, which amounted to $77,054, was met under 
Title 2 funds. Teachers were compensated using TIF 3, 
TIF 4, and general funds for their BCMSSI 
participation. 


Background 


The BCM Summer Science Institute (BCMSSD) is a 
professional development program that offers current 
and effective teaching strategies, in-depth science 
content lessons, assessment, and relevant science 
concepts that are aligned with the Texas Essential 
Knowledge and Skills (TEKS) and the State of Texas 
Assessments of Academic Readiness (STAAR) 
recommended learning objectives (BCM, 2013). This 
program targeted the Houston Independent School 
District (HISD) prekindergarten to eighth-grade 
teachers who taught science in self-contained classes, 
science labs, and regular education classes. The 
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Institute included presentations by scientists and the use 
of inquiry-based, grade-appropriate science lessons. 
Hands-on lessons and activities with online lessons 
were available for workshop participants and other 
teachers. 

New BCMSSI participants attended a two-week 
workshop. Returning elementary- and middle-school 
teachers attended a one-week workshop. In addition, 
teachers were encouraged to attend five follow-up 


Participation in summer science institutes have 
increased the number of teachers providing 
opportunities for their students to conduct full 
scientific inquiry using deep science content and 
process knowledge with numerous opportunities for 
practice (Jeanpiere, Oberhauser & Freeman, 2005). 
Where teachers examined student thinking and their 
implications for instruction, students’ and teachers’ 


science content knowledge and test scores increased 
(Heller, Dahler, Wong, Shinohara & Miratrix, 2012). 

Research has shown that teachers who demonstrated 
improvements in science content knowledge and their 
abilities to analyze science teaching also demonstrated 
higher gains in students’ science content knowledge 
(Roth, Garnier, Chen, Lemmens, Schwille, & Wickler, 
2011). The improvement in students’ science content 
knowledge was also associated with teachers’ 
pedagogical content knowledge about students’ 
thinking, and teaching practices aimed at improving the 
coherence of science content storylines (Roth, et al., 
2011). 

Third- and fourth-grade culturally and linguistically 
diverse students whose teachers had a yearlong science 
professional development program demonstrated 
significant growth in the inquiry abilities of all students. 
Low achieving, low-SES, and LEP students made 
impressive gains (Cueras, Lee, Hart, & Deaktor, 2005). 
A three-year professional development program to 
improve teaching practice significantly raised third- to 
fifth-grade science scores. It consistently decreased the 
achievement gaps among third- and _ fifth-grade 
demographic groups but held steady for the fourth grade 
(Luft, Wong & Ortega, 2009). 

A study designed to measure the influence of inquiry 
science on teacher practices among fourth- and fifth- 
grade teachers in the Los Angeles United School 
District found that there was increased incidence of 
inquiry-based science teaching. The impacts, however, 
were limited to selected features of the inquiry process 
to which the teachers were most frequently exposed 
during the professional development (Griggs, Kelly & 
Geoffrey, 2013). 

Elementary teachers exposed to long-term 
professional development programs demonstrated 
increased content knowledge (Lumpe, Czerniak, Haney 
& Beltyukova, 2012). Teachers in the four-year Rice 
Elementary Model Science Lab (REMSL) increased 
their scientific content knowledge in all four years. In 
the last two years of the program, their gains in science 
content knowledge, use of inquiry based instruction, 
and leadership skills were significantly higher than that 
of the control group _ participants (Deaconu, 
Juskavceric, & Nichol, 2012). Students’ science 
process skills and their content and concept knowledge 
were assessed after their teacher’s participation in an 
elementary STEM professional development program. 
Results showed gains in science process skills, science 
concepts and content knowledge among the general 
education students. Teacher participation in the 
program had statistically significant effects on student 
posttest science scores (Cotabish, Dailey, Ann & 
Hughes, 2013). 

The 2013 and 2014 evaluation of the BCMSSI’s 
impact on elementary and middle school students’ 
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science performance found statistically significant 
increases in the Stanford 10 fifth-grade science in the 
2013 sample and the third and fourth grades in 2014. 
Similar increases occurred at the seventh and eighth 
grades in 2013 and the eighth grade in 2014 among 
students whose teachers participated in the BCMSSI 
program (Holmes, 2013; Serrant, 2014). Linear 
regression modeling indicated that the follow-up or 
additional hours of professional development was the 
strongest predictor of students’ science performance in 
2013 (Holmes, 2013) while students’ previous science 
score was the strongest predictor of their science 
performance in 2014 (Serrant, 2014). 

The purpose of this evaluation is to measure changes 
in BCMSSI teachers’ content knowledge and determine 
the BCMSSI effect on the science performance of 
elementary and middle students whose teachers 
participated in the Institute’s training activities. 
Specifically, the study sought to answer three 
questions: 

1. How did participation in the 2014 BCMSSI impact 

teachers’ science content-knowledge? 

2.How did students whose teachers participated in 

the 2014 BCMSSI perform on the 2015 science 
assessments? 

3. What was the effect of the 2014 BCM Summer PD 


hours on students’ STAAR science performance? 
Methodology 


Data Analysis 

This study involved students whose teachers 
participated in the 2014 summer institute. Only new 
teacher participants were included in the study to 
control for multiple treatment effects. Elementary 
teacher participants were exposed to 60 hours of the 
summer institute’s PD activities, while middle school 
teachers received a maximum of 30 PD hours. Teachers 
were compensated for participation in the Institute 
based on attendance. Professional development hours 
were categorized for analysis based on the U.S. 
Department of Education Institute of Education 
Sciences’ What Works Clearinghouse (WWC) research 
on teacher professional development. Specifically, 
Elementary PD hours were categorized as 1—48.9; 49— 
59.9, and 60 hours. Middle schools were categorized as 
1—29, and 30 hours. WWC research showed that at least 
49 hours of professional development resulted in 21% 
increase in student achievement in_ reading, 
mathematics, science, and social studies (Yoon, 
Duncan, Lee, Scarloss & Shapley, 2009). 


Treatment Effect 

The treatment effects regression adjustment (teffects 
ra) in STATA 13 was used to estimate the effects of the 
BCMSSI PD hours on the STAAR science performance 
of 5"- and 8"-grade students whose teachers completed 
the full 60 PD hours for elementary schools and 30 PD 
hours for middle schools. Fifth and eighth grades were 
selected for student participants because teacher pre- 
post PD tests and students’ STAAR tests were linked to 
the TEKS. In addition, STAAR science tests are offered 
in the 5" and 8" grade only. The regression adjustment 
estimate predicts potential outcomes adjusted for 
covariates assuming that the outcome determinants are 
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few teachers completed them resulting in small sample 
sizes when they were disaggregated by grade. 


Sample 

A total of 239 first- through eighth-grade elementary 
and middle school teachers enrolled as participants in 
the 2014 BCMSSI. Of these, 61 (25.5%) were repeat 
participants. About 48.5% (116 elementary teachers) 
participated in the 60 hours summer sessions. 
Additionally, 62 (25.9%) middle school teachers 
enrolled in the 30-hour summer sessions. 

Only 58 (24.3%) teachers participated in the 20 hours 
of Saturday follow-up sessions. New teachers in the 


known. It uses differences in averages of treatment- 
specific predicted outcomes to estimate treatment 
effects (Cameron & Trivedi, 2005). This study used 
students’ economic, at-risk, special education, limited 
English proficiency, and gifted and _ talented 
identification statuses as the outcome adjustment 
covariates. The treatment group constituted teachers 
who completed the full 60 or 30 BCMSSI PD hours for 
elementary and middle schools, respectively. Teachers 
who did not complete the full PD hours were the 
comparison group. 


Linear Regression Analysis 

Summer PD hours and students’ economic, at risk 
and G/T statuses were analyzed to determine how these 
factors contributed to the variability in students’ science 
scores and, therefore, predict their science performance. 
Collinearity tests were conducted in the selection of the 
variables. The follow-up hours were not included as 


Table 1. Selected Students’ Demographic Characteristics By Grade, HISD, 2015 
Grade Level 

First Second Third Fourth Fifth Sixth Seventh Eighth 

Total students (n) 814 779 1,037 1,401 2,534 2,494 3,218 2,285 
% % % % % % % % 

Asian 8.0 9.3 73 3.6 4.2 3.9 44 4.8 
Black 35.3 35.3 25.6 33.0 26.7 26.7 26.6 30.0 
Hispanic 43.4 40.6 55.8 59.6 63.5 57.7 572 53.5 
White 11.3 12.9 9.7 3.0 5.0 10.4 10.4 10.4 
Non-At-Risk 33.6 40.5 32.4 37.2 40.9 38.6 49.2 39.2 
At-Risk 66.4 59.5 67.6 62.8 59.1 61.4 50.8 60.8 
Non-Special Ed. 96.0 95.5 93.9 93.0 93.2 91.6 92.2 93.1 
Special Ed. 4.0 4.5 6.1 7.0 6.8 8.4 78 6.9 
Non-G/T 83.1 78.0 76.0 83.6 133 83.4 75.4 75.6 
G/T 16.9 22.0 24.0 16.4 24.7 16.6 24.6 24.4 
Non-Economically 30.6 30.1 28.9 17.3 18.5 26.7 32.1 32.3 
Disady. 
Economically Disadv. 69.4 69.9 FLA 82.7 81.5 73.3 67.9 67.7 
Note: These are students whose teachers were new participants in the 2014 BCMSSI 


study were linked to their 2014—2015 science students. 
Table 1 displays the demographic composition by 
grade for these students. The State of Texas 
Assessments of Academic Readiness (STAAR) and 
Iowa Assessments science test data were used to 
measure students’ performance, including the STAAR 
scale score, the proportion of students who met 
Satisfactory performance at the level II phase-in 1 and 
Advanced standard, along with the lowa Assessments 
Normal Curve Equivalent (NCE) and National 
Percentile Rank (NPR). The data satisfied the tests for 
normality, homoscedasticity, and collinearity 
conditions using the Shaphiro-Wilk test, the normal Q- 
Q plot, and the Detrended normal Q-Q plot on the IBM 
Statistical Package for Social Sciences (SPSS) 
software. 


How did participation in the 2014 BCMSSI impact 


teachers’ science content knowledge? 


Gains in the science content knowledge of the 2014 
BCMSSI teacher participants were assessed on four 
multiple-choice tests: introductory elementary, sixth 
grade, seventh grade, and eighth grade. Tests were 
linked to the grade-level science TEKS and validated 
using Cronbach’s alpha (a) (Baylor College of 
Medicine, 2015). Cronbach’s a measures the internal 
consistency of test items to determine how related they 
are as a group (Tavakol & Dennick, 2011). The higher 
the Cronbach’s a, the closer the relatedness and the 
more valid the test will be. With respect to the four tests, 
their Cronbach’s a ranged from 0.39 to 0.68 (Baylor 
College of Medicine, 2015). 

Science content knowledge tests were first 
administered during the first PD session and again on 
the final day and the results reported as the mean 
number of items correct by grade. The results were 
analyzed using a repeated measures design. Table 2 
shows the mean changes and the PD effect on teacher 
content knowledge using Cohen’s d. With the exception 
of 8" grade, all teachers demonstrated significant gains 
in their mean science content knowledge, which ranged 
from a mean difference of 0.44 items correct in the 8" 
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Baylor College of Medicine (2015) also administered 
a post-intervention survey to evaluate teacher 
perceptions and experiences with key program aspects. 
Teacher respondents at the respective grade levels were 
“overwhelmingly positive about the Science Institute. 
They agreed or strongly agreed that the activities were 
well organized and that the materials and resources 
received were useful” (p.18). 


How did the students whose teachers participated in 


the 2014 BCM Summer Institute perform on the 
2015 science assessments? 


Iowa Science 

The BCMSSI _ students’ elementary — science 
performance on the Iowa Assessments ranged from a 
mean NCE of 46.2 (SD = 20.5) in the 4" grade to 56.4 
(SD = 22.5) in the 2™ grade (Table 3). BCMSSI third- 
grade students had a mean NCE of 54.0 (SD = 20.0) 
while first grade had a mean NCE of 47.0 (SD = 25.3). 


grade to 7.53 items correct in the 3" grade. 


Table 3. Elementary students’ 2015 Iowa 
Assessments science performance on the 2014 
BCMSSI teachers 

Mean SD 
Grade n NCE 
1 814 47.0 25.3 
gud 779 56.4 22.5 
a 1,037 54.0 20.0 
4th 1,398 46.2 20.5 
i 2,529 50.7 19.7 


At the middle school level, BCMSSI students’ science 
performance ranged from 45.0 NCEs (SD = 22.2) in the 
6" grade to 49.4 (SD = 20.8) in the 7" grade (Table 4). 
The 8"-grade students had a mean NCE of 48.7 (SD = 


Table 2. BCMSSI teacher content knowledge assessment by grade 
Pretest Posttest Cohen’s 
Grade | n_| Mean | SD Mean | SD Gain 
1 2 12.57 | 3.46 | 14.57 | 2.87 2.00* 0.64 
one 22 11.96 | 3.33 16.50 | 3.71 4.54* 1.32 
3rd 15 13.67 | 2.66 | 21.20 | 2.73 753* 2.89 
gn 14 13.36 | 3.86 16.86 | 2.93 3.50* 1.06 
Se 43 14.51 | 4.10 | 18.05 | 4.71 3.54* 0.81 
6 13 | 23.08 | 3.43 | 27.39 | 1.89 4.31* 1.62 
is 2 19.95 | 2.36 | 22.43 | 2.25 2.48* 1.10 
8th 25 | 20.40 | 2.14 | 20.84 | 6.46 0.44 0.09 
Note: *p < .05 
Cohen’s d: small = .20; medium = .50; and large = .80 
Source: Baylor College of Medicine, 2015 (data only) 


Third-grade teachers in the sample registered the 
highest mean science content knowledge gain followed 
by 2"¢- and 6"-grade teachers. With the exception of the 
gh grade, the effect sizes of the BCMSSI on teacher 
science content knowledge gain ranged from moderate 
to very large, (Cohen’s d) that is, from 0.09 in the 8" 
grade to 2.89 in the 3 grade. 

BCMSSI service provider also measured pre-post 
science teacher self-efficacy using the Self-Efficacy 
Teaching and Knowledge Instrument for Science 
Teachers (SETAKIST). The result showed statistically 
significant gains in self-efficacy on teachers’ personal 
beliefs about their effectiveness in teaching science and 
their pedagogical content knowledge at all grade levels 
in the study (Baylor College of Medicine, 2015). 


21.8). 


Table 4. Middle school students’ 2015 Iowa 
Assessment science performance for 2014 BCMSSI 
teachers 

Grade n Mean SD 

NCE 

6h 2,493 45.0 22.2 
ha 3,218 49.4 20.8 

gh 2,285 48.7 21.8 


Table 5 shows that with the exception of 5"-grade, 
all BCMSSI teachers were exposed to at least 49 PD 
hours. The 3"-grade had the highest proportion of 
BCMSSI students (63.9 percent) who scored at or 
above the 50° NPR on the Iowa science assessments, 
followed by students in the 2™ grade (63.2 percent). 
BCMSSI teachers of students in these two grade levels 
completed between 49 and 60 PD hours. These teachers 


also had the largest content knowledge gain (See Table 


2). 

Table 5. Percent of students of BCMSSI teachers 

who performed at or above the 50" NPR on the 2015 

Iowa science assessments by teachers’ Summer PD 

Hours 

Summer PD Hours 

Grade n 1-48.9 | 49-59.9 60 

1 406 - 33:1 31.5 

one 383 - 63.2 53.6 

ae 588 - 63.9 57.1 

4h 620 - 47.7 36.0 

5° 1,955 40.0 48.4 58.3 

A higher proportion of 1‘ through 4"-grade students 
whose teachers completed 49-59.9 BCMSSI hours 


performed at or above the 50 NPR on the Iowa science 
assessments compared to those whose teachers 
completed all 60 PD hours. While it is uncertain why 
this is the case here, data from the WWC indicates 
significant effect in student’s achievement for at least 
49 PD hours. For teachers who completed all 60 
BCMSS PD hours, 5"-grade had the highest proportion 
of students (58.3%) who performed at or above the 50" 
NPR, followed by 3% grade (57.1 %) and 2"*-grade 
(53.6 %). For teachers who completed all 60 BCMSSI 
PD hours, the 1‘ and 4" grades had the smallest 
proportion of students who performed at or above the 
50 NPR at 31.5% and 36.0%, respectively. 

A higher proportion of middle school students whose 
BCMSSI teachers completed all 30 PD hours (a range 
of 48.2% to 53.1%) performed at or above the 50“ NPR 
on the Iowa science assessments compared to students 
whose teacher completed less hours (a range of 34.6% 
to 47.4%) (Table 6). Eighth-grade students whose 
BCMSSI teachers completed the full 30 PD hours had 
the highest proportion of students performing at or 
above the 50" NPR (53.1%), followed by 7" grade at 
50.0%. 


Table 6. Percent students who performed at or above 
the 50" NPR on the 2015 Iowa science assessment 
of 2014 BCMSSI teachers 


Summer PD Hours 
Grade n 18 -29 30 
6h 2,493 34.6 48.2 
Te 3,218 47.4 50.0 
gh 2,285 40.9 53.1 
STAAR Science 


With respect to the 2015 STAAR science test, fifth- 
grade students whose BCMSSI teachers completed the 
full 60 PD hours had the highest mean scale score 
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(3711) as well as the highest percentage of students who 
met Satisfactory at the Level I] phase-in 1 standard 
(67.1 percent), followed by students whose teachers 
completed 48.9 BCMSSI PD hours or less. These 
students had a mean scale score of 3615 with 58.1% of 
them who met Satisfactory at the Level II phase-in 1 
standard for science (Table 7, Appendix A, p. 8). 

A higher proportion of 5™-grade students whose 
teachers completed the full 60 BCMSSI PD hours 
(9.9%) met the Advanced performance standard on the 
STAAR science test compared to those who completed 
49-59.9 hours (6.6 %) and those who completed 48.9 
hour or less (5.4 %) (Table 7, Appendix A, p. 8). 

Eighth-grade students whose BCMSSI teachers 
completed the full 30 PD hours had a higher mean scale 
score (3659) compared to students whose teacher 
completed less than 30 PD hours (3390) (Table 8, 
Appendix A, p. 8). A higher proportion of 8'-grade 
students whose teacher completed the full 30 BCMSSI 
PD hours (64.8%) met Satisfactory at the Level II 
phase-in | standard on the STAAR science test 
compared to those whose teachers completed less than 
30 PD hours (45.6%), a difference of 19.2 percentage 
points. More than three times as many 8"-grade 
students whose teachers completed the full 30 PD hours 
met Advanced performance standard (17.7%) on the 
2015 STAAR science test compared to students whose 
teachers completed less than 30 hours (5.0 percent). 

Although 8-grade students whose _ teachers 
completed the full 30 hours PD had a mean scale score 
that was lower than the district mean (3659 vs. 3721), a 
higher proportion of those students met Satisfactory at 
the level II phase-in | (64.8% vs. 61.0%). and 
Advanced performance standards (17.7% vs. 14.0%) on 
the 2015 STAAR science test compared to the district. 


What was the effect of the 2014 BCM Summer PD 


hours on_ students’ 2015 STAAR - science 


performance? 


Treatment effects regression adjustment showed the 


effects of 60 summer PD hours of 5-grade and 30 PD 
hours on the 8-grade students’ STAAR science 
performance. The results revealed a significant increase 
of 253 scale score points in the mean scale score of the 
8grade students whose BCMSSI teachers were 
exposed to the full 30 PD hours (Table 9, Appendix A, 
p. 8). The mean scale score for those who were not 
exposed to the full 30 hours (potential outcome mean) 
was 3365 scale score points. Table 2 shows that the gain 
in the 8" grade BCMSSI teachers’ content knowledge 
was small (Cohen’s d = 0.09) but their content 
knowledge appeared to have been high to begin with 
(20.40), second only to 6"-grade teachers (23.08). This 
may help to explain the large effect size. 

The treatment effect of the BCMSSI 60 PD hours on 
the 5"-grade science performance was small (17 scale 


score points) and statistically insignificant (p > .524) 
(Table 10, Appendix A, p. 9). The potential outcome 
mean using STATA teffects ra, indicated that for 
students whose teachers were not exposed to the full 60 
PD hours their expected scale score was 3589 (Table 
10, Appendix A, p. 9). 

Based on regression analysis results, the summer PD 
hours were responsible for small variations in the Iowa 
Assessment science NCE for 1* through 8" grade. The 
variations were significant at the p < .0001 level for 2"4, 
and 6" through 8" grades (Table 11 and Table 12, 
Appendix A, p. 9). Too few teachers completed the 
follow-up sessions to conduct any meaningful data 
analyses. 

Students’ at-risk and economically-disadvantaged 
statuses were negative predictors of the 2015 STAAR 
science mean scale score at the 1‘ through 8"-grade 
levels for BCMSSI students. Students’ at-risk status 
(the largest prediction of the two) predicted between 
27.2% and 44.1% of variations in the science NCE for 
all grades in the study. G/T identification was a positive 
predictor at all grade levels in the study, predicting 
between 24.3% and 34.3% variations in the BCMSSI 
teacher’ students mean NCE (Table 11 and Table 12). 


Discussion 


Results of this evaluation showed that teacher content 
knowledge increased for all teacher groups in the study, 
ranging from mean difference of 0.44 items correct in 
the 8"-grade to 7.53 items correct in the 3" grade. The 
PD effect, using Cohen’s d, was moderate or large for 
all groups except 8"-grade but 8""-grade teacher content 
knowledge was high to begin with. Cohen’s d is a 
measure of the effect sizes of an intervention using 
standard deviation units (Pallant, 2013). In this study, 
Cohen’s d ranged from 2.89 in the 3" grade to 0.09 at 
the 8" grade. The gains were statistically significant (p 
< .05) at the elementary school level. The 2"*- and 3"¢- 
grade students whose teachers participated in the 
BCMSSI had the highest mean science NCE. They also 
had the highest percentage of students scoring at or 
above the 50% NPR. Their teachers registered the 
largest science content knowledge gains on pre-post 
BCMSSI tests that were aligned to the TEKS. These 
gains may have been the pivotal factor in the program 
impact. 

A higher proportion of fifth through eighth-grade 
students whose teachers completed the full PD hours 
performed at or above the 50" NPR on the Iowa 
Assessment science tests compared to students whose 
teachers did not complete the full PD hours. Similarly, 
a higher proportion of 5" and 8" grade students whose 
teachers completed the full PD hours met Satisfactory 
at the Level II phase-in 1 and Advanced performance 
standard of the 2015 STAAR science tests. 
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The treatment effect of the full 60 PD hours on the 
5""_grade science test was small (17 scales core points) 
and not statistically significant (p = .524). The 
treatment effect of the full 30 PD hours on 8'"-grade 
science performance was large (253 scale score points) 
and statistically significant (p <.0001). Based on these 
findings, 422 (18%) additional 8"-grade students might 
have met Satisfactory at the Level II phase-in | standard 
on the 2015 STAAR science test had their teachers 
completed the full 60 BCMSSI PD hours. This 
demonstrates the effectiveness of the BCMSSI at the 
8-grade level. This result is critical because 8" grade 
marks the end of middle school and the transition to 
high school. More may have to be done to increase 
teacher completion of PD and the cost-effectiveness of 
the program since the service provider is contracted. 
Teachers are already being compensated, financially, 
for participation in the program. 

Greater effort needs to be made to promote teacher 
commitment to program completion once enrolled. 
Compensation may have to be tied to completion rather 
than participation. Continued instructional focus should 
be directed towards at-risk and economically- 
disadvantaged students. 


References 


Baylor College of Medicine (2013). Teachers become 
the students at Baylor Summer Science Institute. 
Retrieved from Momentum: Moving forward in 
education, healthcare, research, and community: 
http://momentumblog.bcem.edu/2013/08/13/teachers- 
become-the-students-at-baylors-summer-science- 
institute/ 

Baylor College of Medicine (2015). Baylor Summer 
Science Institutes and Baylor Saturday Science 
Series, Final Report. Baylor College of Medicine: 
Center for educational Outreach 

Cameron, C. A., & Trivedi, P. K. (2005). 
Microeconomics: Methods and _ applications. 
Cambridge: Cambridge University Press. 

Cotabish, A., Dailey, D., Ann, R., & Hughes, G. (2013). 
The effects of a STEM intervention on elementary 
students science knowledge and_ skills. School 
Science and Mathematics 113 (5), 215-226. 

Cueras, P., Lee, O., Hart, J., & Deaktor, R. (2005). 
Improving science inquiry with elementary students 
of diverse backgrounds. Journal of Research in 
Science teaching 42, 337-357. 

Deaconu, D. V., Juskavceric, M., & Nichol, C. (2012). 
A multi-year study of the impact of the Rice Model 
teacher professional development on elementary 
science teachers. International Journal of Science 
Education 34 (6), 855-877. 

Griggs, J., Kelly, K. A., & Geoffrey, B. D. (2013). 
Effects of two scientific inquiry professional 


development interventions on teaching practice. 
Education Evaluation and Policy Analysis 36 (1) , 
38-56. 

Heller, J. I., Dahler, K. R., Wong, N., Shinohara, L. W., 
& Miratrix, L. W. (2012). Differential effects of three 
professional development models on_ teacher 
knowledge and student achievement in elementary 
science. Journal of Science Research 49, 333-362. 

Holmes, V. (2013). The impact of the Baylor College of 
Medicine (BCM) 2011 and 2012 Summer Institute on 
elementary- and middle-school students science 
performance. HISD Research and Accountability 
Department Evaluation Report. 8 (1), 1-9 

Jeanpiere, B., Oberhauser, K., & Freeman, C. (2005). 
Characteristics of professional development that 
effect change in secondary science teachers' 
classroom practices. Journal of Research in Science 
Education 42 (6), 668-690. 

Luft, J. A., Wong, S., & Ortega, I. (2009). The National 
Science Teachers Association (NSTA) state science 
education survey. Arlington. VA: NSTA. 

Lumpe, A., Czerniak, C., Haney, J., & Beltyukova, S. 
(2012). Beliefs about teaching science: The 
relationships between elementary __ teachers' 
participation in professional development and 
student achievement. International Journal of 
Science Education 34 (2), 153-166. 

National Science Teachers Association. (2002). NSTA 
Position Statement. Retrieved February 25, 2014, 
from National Science Teachers Association 
http://www.nsta.org/about/positions/elementary.asp 
x 

Pallant, J. (2013) A step by step guide to using IBM 
SPSS: Survival manual, 5 edition. Berkshire, 
England: Open University Press. 

Pruitt, S. L., & Wallace, C. S. (2012). The effect of a 
State Department of Education Teacher Mentor 
Initiative on science achievement. Journal of Science 
Education 23 (4), 367-385. 

Roth, K. J., Garnier, H. E., Chen, C., Lemmens, M., 
Schwille, K., & Wickler, N. I. (2011). Videotaped 
lesson analysis: Effective science PD for teacher and 
student learning. Journal of Research in Science 
Teaching 48 (2), 117-148. 


Baylor Science Lab, 2014-2015 


Serrant, T. D. (2014). The effect of the Baylor College 
of Medicine Summer Science Institute on HISD 
elementary and middle school students’ science 
performance, 2014-2015. HISD Research and 
Accountability Department Evaluation Report 8 (1), 
1-9 

Texas Education Agency. (2011). Texas Education 
Agency Best Practices Clearinghouse. Retrieved 
April 03, 2014, from How to interpret effect sizes: 
http://www.tea.state.tx.us/Best_Practice_Standards/ 
How_To_Interpret_Effect_Sizes.aspx 

Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & 
Shapley, K. L. (2009). Reviewing the evidence on 
how teacher professional development affects student 
achievement (Issues & Answers, REI 2007-No033). 
Washington DC: U.S. Department of Education, 
Institute of Education Sciences, National Center for 
Education Evaluation and Regional Assistance, 
Regional Education Laboaratory Southwest. 


For additional information contact the HISD 
Department of Research and Accountability at 713- 


556-6700 or e-mail Research @houstonisd.org. 


Appendix A 


Baylor Science Lab, 2014-2015 


Table 7. 2015 STAAR Mean Science Scale Score and Percent Who Met Standards, BCMSSI Fifth 
Grade 


*5' Grade Mean Std. Dev. Level I: Satisfactory Advanced 
Scale Score Phase-in | Standard Performance 
(n = 2561) (%) Standard (%) 
1—48.9 PD hours 3615 504 58.1 5.4 
49-59.9 PD hours 3557 564 55.3 6.6 
60 PD hours 3711 554 67.1 9.9 
Note: *5" grade — “Satisfactory” Scale Score range : 3500-4304; “Advanced” Scale Score : 4402-5613 
(Texas Education Agency, 2015) 


Table 8. 2015 STAAR Mean Science Scale Score and Percent Who Met Standards, BCMSSI Eighth 
Grade 


*8'" Grade Mean Std. Dev. Level II: Satisfactory Advanced 
Scale Score Phase-in | Standard Performance 

(n = 2361) (%) Standard (%) 

18—29 PD hrs. 3390 749 45.6 5.0 

30 PD hrs. 3659 998 64.8 17.7 


Note: *8" grade — “Satisfactory” Scale Score range : 3500-4345; “Advanced” Scale Score: 4406-6180 
(Texas Education Agency, 2015) 


Table 9. Treatment effect of 2014 BCMSSI PD Hours on 2105 8'°-Grade STAAR Science Performance 


Science Scale Score Coefficient | Robust Z p>z [95% Confidence 
Std. Interval] 
Error 
Average Treatment Effect (ATE) 
‘Summer PD Hours 
(2 vs. 1) 252.9 54.4 4.7 0.000* 146.3 3595 
Potential Outcome Mean 
‘Summer PD hours 
1 3364.8 50.1 67.2 0.000* 3266.7 3462.9 
*p < .0001 Note: '1=18-29PD hours; 2= 30 PD hours 


oo 
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Table 10. Treatment effect of 2014 BCMSSI PD Hours on 2015 5"*-Grade STAAR Science 
Performance 


Science Scale Score Coefficient | Robust Z p>z [95% Confidence 
Std. Err Interval] 
Average Treatment Effect 
(ATE) 
‘Summer hours 16.5 25.9 0.64 0.524 -34.2 67.3 
(2 vs. 1 ) 


Potential Outcome mean 
‘Summer Hours 
1 3589.0 21.2 169.5 .000* 3547.5 3630.5 


*p < .0001 Note: 'l = 1-59.9PD hours; 2 = 60 PD hours 


Table 11. Linear Regression Model of Iowa Assessments Science NCE (1° to 4 Grades), 2015 


2015 Iowa 2015 Iowa 2015 Iowa 2015 Iowa 
Variable Assessment 1* Assessment 2" Assessment 3" Assessment 4" 
Grade Science Grade Science Grade Science Grade Science 
NCEs NCEs NCEs NCEs 
n= R= n= R?= n= R? n= R?= 
815 30.0% 781 30.6% 1,037 =35.8% 1,042 30.3% 
P B P P P 
Economically Disadv. -.220 .000* -.19.1 .000* -.121 .000* -.110 .000* 
Summer PD -.058 500 -.112 .000* 005 848 -.012 585 
At risk status -.282 .000* -.272 .000* -.361 .000* -.338 .000* 
G/T 243 .000* 271 .00*0 305 .000* 301 .000* 


Note: *p <.0001 


Table 12. Linear Regression Model of Iowa Assessments Science NCE (5' to 8 Grades), 2015 


2015 Iowa 2015 Iowa 2015 Iowa 2015 Iowa 
Variable Assessment 5" Assessment 6" Assessment 7" Assessment 8" 
Grade Science Grade Science Grade Science Grade Science 
NCEs NCEs NCEs NCEs 
n= R= n= R= n= R?= n= R?= 
2,535 | 49.1% | 2,501 52.3% 3,223 52.3% 2,288 48.3% 
B P B P B P B P 
Economically Disadv. -.149 | .000* -.100 .000* -.101 000* -.101 .000* 
Summer PD .019 224 112 .000* .067 000* .072 .000* 
At risk status -.394 | .000* -.421 .000* -.44] .000* -.377 .000* 
G/T .264 | .000* ..304 .000* 335 .000* 343 .000* 


Note: *p<.0001 


